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About  this  series 

This  white  paper  is  the  fifth  in  a  five-part  series  dedicated  to 
examining  problems  organizations  encounter  when  operating  in 
multimodel  environments  and  the  current  process  improvement 
approaches  such  organizations  need  to  consider.  It  addresses  the 
implementation  challenges  faced  by  process  improvement 
professionals  in  multimodel  environments,  where  it  becomes 
necessary  to  coordinate  roles  and  responsibilities  of  the 
champions  for  different  technologies,  to  integrate  and  coordinate 
training,  to  optimize  audits  and  appraisals,  and  develop  an 
integrated  approach  to  project  portfolio  management. 


The  rest  of  this  series  addresses,  in  more  detail,  each  phase  of  the 

reasoning  framework  for  technology  harmonization  in  a  multimodel  environment: 

•  The  1st  white  paper  addresses  the  benefits  of  a  harmonized  approach  when  implementing  more  than  one 
improvement  model,  standard,  or  other  technology  and  provides  a  high-level  description  and  underlying 
paradigms  of  a  reasoning  framework  for  technology  harmonization. 

•  The  2nd  white  paper  examines  the  approaches  needed  in  technology  selection  including  a  strategic  taxonomy, 
the  decision  authorities  associated  with  that  selection  at  all  levels  in  the  organization,  and  considerations  for 
thoughtful  sequencing  of  implementation  in  alignment  with  the  organizations'  mission,  goals  and  objectives. 

•  The  3rd  white  paper  examines  technology  composition  in  relation  to  the  concepts  introduced  in  the  previous 
white  papers;  a  proposed  element  classification  taxonomy  to  make  technology  integration  effective  in  practice; 
and  the  role  of  technology  structures,  granularity  and  mappings  in  technology  composition. 

•  The  4th  white  paper  examines  the  current  state  of  the  practice  for  defining  process  architecture  in  a  multimodel 
environment,  methods  and  techniques  used  for  architecture  development,  and  underlying  questions  for  a 
research  agenda  that  examines  the  relationship  of  technology  strategy  and  composition  to  process  architecture 
as  well  as  the  interoperability  and  architectural  features  of  different  process  technologies. 
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A  multimodel  process  improvement  environment  has  significant  implications  for  an 
organization’s  improvement  infrastructure  and  for  the  way  in  which  new 
technologies 1  or  changes  to  the  existing  technology  mix  are  deployed  in  the 
organization.  This  white  paper  addresses  the  implementation  challenges  faced  by 
process  improvement  professionals  in  multimodel  environments: 

•  Establish  a  coordinated  process  improvement  infrastructure 

-  coordinate  roles  and  responsibilities  of  the  champions  for  different  technologies 

-  integrate  and  coordinate  training 

•  Develop  of  an  integrated  approach  to  improvement  project  portfolio  management 

•  Optimize  of  audits  and  appraisals 

•  Establish  aligned  measurement  systems  (which  provides  additional  benefits,  such 
as  integration  and  governance  motivation) 


PROCESS  IMPROVEMENT  INFRASTRUCTURES 

The  organizational  structures  established  to  support  the  achievement  of 
organizational  excellence  are  as  varied  and  diverse  as  the  organizations  themselves. 
This  diversity  is,  in  general,  positive,  reflecting  the  adaptation  of  organizational 
structure  to  the  specific  business  context.  While  the  organization  structures  for 
process  improvement  are  diverse,  there  are  common  patterns  that  have  proven 
themselves  in  many  improvement  initiatives  and  are  recognizable  in  most 
organizations  and  indeed  in  many  improvement  technologies.  We  will  discuss  these 
common  patterns  and  the  adjustments  needed  for  successful  implementation  in  a 
multimodel  context. 

There  are  three  common  organizational  roles  in  structures  for  process  and  product 
improvement.  In  relation  to  individual  improvement  technologies  being  adapted, 
these  are: 

•  Senior  management  who  provide  sponsorship,  budget,  and  strategic  direction  to 
the  improvement  efforts  of  the  organization.  This  may  be  in  the  form  of  a  steering 
group  or  it  may  be  built  into  the  responsibilities  of  individual  managers. 

•  Personnel  to  facilitate  and  co-ordinate  process  and  product  improvement  in  the 
organization.  Often  personnel  are  organized  around  or  charged  with  a  particular 
improvement  technology. 

•  Improvement  teams  that  are  temporary  in  nature,  implementing  specific 
improvement  as  directed  by  the  senior  management  and  the  improvement 
personnel. 

Ideally,  these  three  organizational  roles  are  staffed  with  influential  and  respected 
individuals  who  are  experienced  in  the  operative  realities  of  the  organizations 
business.  In  larger  organizations,  these  roles  may  be  repeated  in  a  complex  multi¬ 
layered  hierarchy  that  also  might  be  very  geographically  diverse.  Of  course,  these  are 

1  In  this  series  of  white  papers,  we  use  the  terms  improvement  technologies ,  technologies,  or  models 
somewhat  interchangeably  as  shorthand  when  we  are  referring  in  general  to  the  long  list  of 
reference  models,  standards,  best  practices,  regulatory  policies,  and  other  types  of  practice-based 
improvement  technologies  that  an  organization  may  use  simultaneously. 
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just  the  basic,  common  components.  The  reality  is  somewhat  more  complex.  Some 
improvement  technologies  provide  guidance  for  the  infrastructure  necessary  to 
implement  and  sustain  improvement  in  an  organization: 

•  The  Carnegie  Mellon®  Software  Engineering  Institute’s  Capability  Maturity 
Model®  Integration  (CMMI®)  recommends  the  establishment  of  management 
steering  committees,  process  groups,  process  action  teams  and  process  owners  as 
best  practice  when  focusing  on  process  at  an  organizational  level  [Chrissis  2006]. 

•  Six  Sigma  infrastructures  are  a  core  aspect  of  its  codified  deployment  element, 
including  adoption  decisions  and  advocacy  from  executive  level,  champions  who 
select  projects  and  remove  barriers,  and  implementation  improvement  through  a 
network  of  trained  experts  called  “Black  Belts,”  or  “Belts,”  for  short.  Belts 
shoulder  the  majority  of  improvement  project  leadership,  with  Master  Black  Belts 
taking  responsibility  for  large  or  very  complex  (or  highly  specialized)  projects. 
Green  Belts  may  lead  smaller  projects,  but  often  serve  on  project  teams,  along 
with  subject  matter  experts,  domain  experts  and  other  cross -functional 
representatives. 

Despite  the  similarity  in  structures  typically  established  by  different  improvement 
technologies,  the  reality  is  that  champions  of  each  technology  establish  their  own 
improvement  organization,  with  duplication  of  similar  functions  and  with  consequent 
cost  duplication  to  the  organization  as  a  whole. 

Organizations  often  arrive  at  the  realization  that  they  are  in  a  multimodel 
environment  over  a  period  of  several  years.  For  example,  the  engineering  group  starts 
a  CMMI-based  improvement  and  establishes  the  typical  CMMI  infrastructure  to 
implement  change.  A  new  executive  level  manager  arrives  from  an  organization 
using  Six  Sigma  and  begins  a  corporate  Six  Sigma  initiative,  including  the 
organizational  structure  needed  to  support  it.  It  is  often  only  when  conflict  and 
friction  between  these  (and  potentially  many  more)  groups  surfaces  that  the 
organization  realizes  it  is  has  parallel,  overlapping,  and  competing  organizational 
structures.  All  of  those  structures  generate  costs  and  all  try  to  improve  some  aspect  of 
the  organization’s  performance.  The  pain  these  circumstances  cause  argues  strongly 
for  a  harmonization  of  the  structures  supporting  organizational  improvement. 

Our  research  has  shown  that  organizations  succeeding  with  multimodel  environments 
work  to  optimize  their  improvement  infrastructures.  Such  optimization  may  include: 

•  the  establishment  of  integrated  structures  across  multiple  improvement 
technologies 

•  a  systematic  distribution  of  responsibilities  across  improvement  technology 
structures  for  strategic,  tactical  or  domain  related  improvement  or  a  combination 
of  these 


Carnegie  Mellon  and  CMMI  are  registered  in  the  U.S.  Patent  and  Trademark  Office  by  Carnegie 
Mellon  University. 
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Shared  and  coordinated  roles  and  responsibilities 


Organizations  in  multimodel  improvement  environments  need  to  consider  having  the 
respective  improvement  technology  experts  work  in  the  same  teams,  share  roles  or 
identify  other  means  of  establishing  a  seamless  partnership.  The  objective  is  for  the 
champions  and  implementers  of  different  technologies  to  have  a  shared  sense  of 
organizational  mission  and  goals  and  a  shared  sense  of  responsibility  for  establishing 
a  successful  integrated  process  improvement  program  that  achieves  all  of  their 
objectives 

There  is  a  strong  case  for  a  systematic  integration  of  change  agents  in  cross¬ 
technology  improvement  structures,  whether  these  are  permanent  or  temporary  in 
nature.  This  cross-technology  sharing  and  coordination  of  roles  may  take  different 
forms  both  from  the  type  of  cooperation  established  and  the  temporal  nature  of  the 
structures  established.  For  example,  where  change  control  boards  are  considering  the 
introduction  of  new  or  changed  technologies  into  the  technology  mix  implemented  in 
an  organization,  these  bodies  should  typically  be  permanent  and  staffed  with  cross - 
technology  experts  who  are  empowered  to  make  relevant  decisions.  In  contrast, 
teams  of  cross-technology  experts,  assembled  for  the  implementation  of  specific 
improvements  are  often  of  temporary  nature. 

A  good  example  of  how  roles  can  be  shared  and  coordinated  is  afforded  by  two 
technologies  we  have  studied  in  practice:  CMMI  and  Six  Sigma.  For  example,  when 
looking  for  candidates  to  train  as  Six  Sigma  Belts,  an  organization  should  consider 
Engineering  Process  Group  members.  Six  Sigma  champions  need  to  understand  (or 
at  least  have  awareness)  of  CMMI  and  other  discipline  specific  technologies. 

Another  consideration  for  identifying  shared  roles  and  responsibilities  is  to  leverage 
the  codified  best  practices  available  for  just  this  sort  of  purpose.  As  an  example,  the 
CMMI  is  a  useful  source  of  good  practices  for  the  establishment  and  maintenance  of 
effective  integrated  teams  [CMMI-DEV  vl.2].  To  successfully  integrate  technologies 
and  improvement  infrastructures  within  an  organization,  decision  makers  need  to 
consider  how  to  develop  a  sense  of  shared  vision  in  those  with  responsibility  for 
improving  organizational  excellence.  The  development  of  a  common  “understanding 
of  the  organizational  mission,  goals,  expectations  and  constraints  allows  the  project 
to  align  its  direction,  activities,  and  shared  vision  with  the  organization  and  helps 
create  a  common  purpose  within  which  project  activities  can  be  coordinated”  [Siviy 
2007], 

The  CMMI  stresses  that  teams  and  functional  experts  cannot  operate  effectively  in 
isolation.  Developed  specifically  for  teams  developing  products,  the  CMMI  for 
Development  constellation  using  integrated  product  and  process  development 
practices  (IPPD)  can  equally  well  be  applied  to  those  having  to  work  together 
effectively  in  integrating  improvement  technologies.  The  CMMI  also  stresses  that  an 
effective  communication  strategy  is  critical  to  implementing  and  focusing  on  the 
shared  vision.  This  aspect,  too,  is  applicable  in  the  context  we  are  addressing.  In 
addition,  the  CMMI  gives  useful  guidance  on  how  to  establish  and  maintain  relevant 
team  structures  when  working  in  a  collaborative  and  coordinated  manner,  which  we 
believe  is  also  critical  to  success  in  process  improvement  in  a  multimodel  context. 
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Integrated  and  coordinated  training 


Having  translated  our  organizations  mission  to  inform  improvement  strategy  (see  the 
2nd  white  paper  in  this  series)  and  established  coordinated  organizational  structures  to 
support  multimodel  process  improvement,  we  need  to  consider  how  to  improve  the 
operational  effectiveness  of  the  resultant  shared  roles  and  responsibilities  and 
associated  collaborations.  Integrated  and  coordinated  training  is  one  of  the 
mechanisms  we  recommend  to  help  improve  the  effectiveness. 

Inte grated/ coordinating  training  might  include  such  things  as 

•  All  change  agents,  champions  and  improvemen  t  professionals  receive  awareness 
level  training  about  all  selected  technologies 

It  is  vitally  important  for  improvement  technology  experts,  especially  the  change 
agents  for  each  improvement  technology,  to  development  an  adequate 
understanding  of  the  details  of  the  technologies  that  need  to  be  integrated  and 
interoperable  within  an  organization.  There  are  two  pertinent  reasons  for  requiring 
this  cross -technology  awareness  and  competence.  This  knowledge  is  essential  if 
change  agents  are  to  work  together  effectively  in  the  mission  translation  and 
model  composition  tasks  explained  in  the  3rd  white  paper  of  this  series.  While 
considering,  for  example,  the  impact  of  change  proposals,  members  of  a  change 
control  board  need  to  be  aware  of  the  manner  in  which  currently  integrated 
technologies  work  together  and  the  consequences  of  any  proposed  changes.  This 
requires  the  establishment  of  effective  cross-technology  team  structures  and 
cross-technology  competence  among  the  different  technology  change  agents.  In 
addition,  without  an  appreciation  for  the  affinity  and  granularity  relationships  of 
the  different  technologies,  an  effective  coordination  and  cooperation  of  different 
technology  change  agent  will  be  difficult  to  achieve. 

•  Selected  improvement  professionals  receive  in  -depth  technology  training  in  each 
selected  technology;  consider  training  some  individuals  in  more  than  one 
technology  to  help  bridge  communications  gaps,  understand  the  technical  links 
between  technologies,  and  so  on. 

There  is  still  a  need  to  train  improvement  professionals  in  the  organization  in  the 
individual  source  technologies  and  to  provide  training  in  how  the  source 
technologies  are  related  and  connected.  The  improvement  professionals  need  to 
develop  a  deep  understanding  of  these  source  technologies,  plus,  the  organization 
should  provide  supplemental  training  that  focuses  on  important  strategic  and 
tactical  relationships  between  the  source  technologies.  Organizations  may  develop 
such  a  training  delivery  competence  in-house,  supported  by  a  growing  research 
literature  in  this  area.  Some  outsourced  training  provisioning  is  also  beginning  to 
become  available  [Siviy  2007]. 
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•  All  system  and  software  developers  receive  awareness  or  in-depth  training  about 
selected  technologies;  this  training  is  primarily  in  the  organizations  standard 
process,  but  likely  includes  discipline-specific,  tactical  technologies. 
Organizations  that  have  established  an  integrated  process  architecture,  which 
places  focus  on  the  executed  processes,  are  acutely  aware  that  operative  staff 
needs  to  be  trained  in  the  executed  process  and  not  in  the  various  source 
technologies  that  were  used  as  input  for  the  executed  process.  The  cost  savings 
organizations  can  realize  by  training  their  project  and  operative  staff  in  the 
implemented  version  of  the  organizations  processes — not  in  a  broad  range  of 
source  technologies — are  significant.  That  is  not  to  say  that  orientation  and 
awareness  training  for  specific  roles  that  need  to  understand  the  relationship  of 
the  changes  planned  to  the  relevant  source  models  should  not  be  undertaken. 
Management  roles  that  need  to  support  the  rollout  of  changes  in  the  organization 
through  a  focused  customization  of  the  rewards  and  recognition  system  should 
receive  this  kind  of  training,  for  example.  Overall,  training  large  numbers  of  staff 
in  the  details  of  the  source  technologies  may  lead  to  a  broad  understanding  of  the 
terminologies  used  in  these  technologies  in  the  organization,  but  these  benefits 
must  be  balanced  against  very  considerable  costs  in  most  cases.  As  a  rule,  large 
scale  training  in  the  source  technologies  is  not  recommended. 

To  achieve  cross-technology  awareness  and  competence  at  each  of  the  described 
levels,  organizations  need  to  find  effective  ways  of  cross-training  experts  in 
individual  improvement  technologies  from  the  suite  of  improvement  technologies 
relevant  to  the  organization.  The  competence  management  system  in  organizations 
needs  to  adjust  its  objectives  and  approaches  to  ensure  that  individual  experts  in 
single  technologies  receive  cross-training  in  other  relevant  technologies  to  the  degree 
required  for  the  roles  they  have  in  the  organization. 


Pairing  of  CMMI  and  Six  Sigma 


Some  CMMI  experts  may  require  training  in  Six  Sigma  to  help  them  understand  how  Six  Sigma  helps  achieve  a 
quantitatively  managed  process,  while  others  will  need  a  level  of  training  allowing  them  to  work  on  teams 
charged  with  implementing  quantitatively  managed  processes.  Six  Sigma  Black  Belts  may  require  introductory 
training  in  CMMI  to  understand  the  architecture  of  the  model  and  to  understand  the  software  and  systems 
engineering  domain  the  model  is  typically  applied  in.  Other  Six  Sigma  experts  may  require  training  that  is  more 
fundamental  in  the  systems  engineering  domain.  Both  Six  Sigma  and  CMMI  experts  may  require  a  common 
CMMI-Six  relationship  training.  This  type  of  supplementary  cross-domain  training  is  common  practice  in  some 
organizations.  The  types  of  training  consideration  just  discussed  for  Six  Sigma  and  CMMI  are  by  no  means 
comprehensive  (see  Siviy  2007  for  a  more  detailed  discussion).  These  inter-relationships  and  related  cross¬ 
training  should  be  addressed  for  all  relevant  improvement  technologies  in  the  organization.  Not  addressing 
these  in  training  or  not  understanding  the  inter-relationships  adequately,  before  developing  or  outsourcing 
cross-training,  can  be  costly  in  terms  of  the  expenditure  wasted.  Additional  costs  may  arise  down  the  line  as  a 
consequence  of  decisions  made  based  upon  seriously  deficient  cross-technology  competence  and  awareness. 
There  is  therefore  a  strong  business  case  to  be  made  for  a  systematic  and  professional  approach  to  the  cross - 
training  of  experts  from  the  individual  improvement  technologies  implemented  in  an  organization. 
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COORDINATED  IMPROVEMENT  PROJECT 
PORTFOLIO  MANAGEMENT 


In  their  article  “How  the  Learning  Organization  Manages  Change”  Ronald  Recardo, 
Kathleen  Molloy,  and  James  Pellegrino  make  the  important  observation  that  the 
translation  of  organizational  goals  and  related  metrics  to  the  teams  and  individuals 
responsible  for  effecting  change  in  the  organization  is  one  of  the  most  significant 
barriers  to  successful  process  improvement  [Recardo  et  al.  2007].  Those  charged 
with  managing  and  driving  process  improvement  in  the  organization  have  to  ensure 
traceability  between  the  organizations’  mission  and  goals  and  the  improvement 
activities  planned  and  executed  in  the  organization.  In  a  multimodel  environment  the 
mission  translation  activities  are  a  critical  activity,  needing  to  guide  technology 
selection,  informing  how  the  different  technologies  can  be  combined  effectively  and 
providing  key  inputs  to  the  selection  of  success  measures  for  the  improvement 
efforts.  A  systematic  approach  to  these  activities  is  advisable,  as  this  will  help  ensure 
objectivity  of  the  staff  performing  these  activities.  In  addition  mission  translation 
critically  informs  the  decisions  required  to  ensure  the  design  of  an  effective 
coordination  of  improvement  project  portfolio  management.  Even  where  mission 
translation  and  goal  decomposition  is  performed,  the  number  of  potential 
improvement  related  tasks  or  projects  from  the  various  source  technologies  addressed 
in  an  organization  can  be  quite  large.  There  are,  in  addition,  many  other  sources  of 
improvement  suggestions,  including  ideas  generated  from  lessons  learned  feedbacks 
from  operational  projects,  appraisal  results  and  lists  of  items  from  various  types  of 
gap  analyses.  Thus  an  organization  will  typically  have  several  extensive  lists  of  items 
that  might  be  improved  and  these  may  be  separate  and  competing.  A  systematic 
coordination  and  prioritization  of  these  lists  and  the  improvement  projects  derived 
from  them  is  needed.  The  use  of  Six  Sigma  and  other  methodologies  to  understand 
the  voice  of  the  customer,  the  voice  of  the  business  and  the  voice  of  the  process  helps 
inform  the  prioritization  undertaken.  This  prioritization  should  include  enabling 
projects  which  can  be  better  justified  with  an  understanding  of  the  “voices”  relevant 
to  the  organization. 

These  mission  translation  activities  are  prerequisites  for  an  effective  implementation 
of  improvement  projects  at  different  levels  in  the  organization,  addressing  individual 
or  integrated  improvement  technologies  across  an  extended  period.  The  requisite 
sequencing,  major  dependencies  and  constraints  of  inter-related  improvement  project 
need  to  be  addressed  effectively  by  those  charged  with  managing  improvement  in  the 
organization.  A  seamless  synchronization  of  improvement  projects  that  are  aligned 
with  the  mission,  goals  and  objectives  of  the  organization  necessitates  the 
establishment  of  a  coordinated  improvement  project  portfolio  management.  We 
define  coordinated  improvement  project  portfolio  management  as  the  set  of 
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approaches  used  for  prioritizing  and  collectively  managing  all  current  or  future 
improvement  projects  based  on  characteristics  derived  from  a  sound  understanding  of 
the  organization’s  mission.  Its  goal  is  thus  to  determine  the  optimal  mix  and 
sequencing  of  proposed  projects  to  best  achieve  the  organization's  misson.  The  risks 
associated  with  not  implementing  improvement  project  portfolio  management  are 
manifold,  including  internal  competition  for  scarce  improvement  related  funding,  the 
establishment  of  divergent  and  competing  improvement  technology  project  structures 
including  different  reporting  and  communication  channels.  This  risk  is  heightened 
where  individual  improvement  technology  initiatives  engage  in  gap  analysis  of  the 
organization  against  their  own  technology.  If  an  overarching  and  effective 
improvement  project  portfolio  management  is  not  in  place,  the  activities  resulting 
from  the  gap  analysis  will  be  uncoordinated  with  other  activities  in  the  organization. 
Properly  embedded  in  the  improvement  project  portfolio  management  system 
however,  the  individual  gap  analyses  provide  an  important  input  to  the  totality  of  all 
improvement  activities  in  the  organization.  Thus  top-down  and  bottom-up 
approaches  to  the  identification  of  improvement  activities  is  supported  by  the 
implementation  of  a  improvement  project  portfolio  management. 

In  a  non-harmonized  approach,  improvement  projects  can  be  isolated  from  each  other 
and  from  the  overall  organizational  mission.  Goal  decomposition,  as  we  said  in  the 
2nd  white  paper  of  this  series,  gives  all  improvement  projects  a  line  of  sight  to  the 
topmost  organizational  goals  and  an  explicit  relationship  to  one  another.  Such 
methods  can  be  incoiporated  into  the  organizational  standard  processes  for 
identifying  and  defining  projects.  The  integrated  improvement  project  portfolio  that 
arises  from  a  harmonized  multimodel  approach  also  gives  the  organization  an 
understanding  of  the  role  of  “enabling”  projects  in  establishing  processes  and 
measures  needed  for  subsequent  improvement  efforts  that  have  direct  bottom  line 
benefit.  Enabling  projects  establish  required  infrastructure,  processes  or  measurement 
systems  that  subsequent  improvement  projects  will  utilize,  but  themselves  have  no 
direct  contribution  to  bottom-line  savings.  The  economic  justification  of  such 
enabling  projects  is  easier  in  a  harmonized  multimodel  approach,  where  the  inter¬ 
relationships  between  improvement  technologies  and  the  improvement  projects 
launched  to  implement  them  are  well  understood. 

Enablers  for  a  coordinated  improvement  portfolio  management  include  the  shared 
roles,  improvement  infrastructure  and  cross-training  discussed  earlier  in  this  paper. 
With  these  in  place  we  have  a  skilled,  integrated  and  synchronized  improvement 
workforce  that  are  well  equipped  to  manage  the  balancing  of  different  interest,  needs 
and  expectation  of  the  many  and  varied  stakeholders  an  organization  has  in  relation 
to  improvement  projects. 
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COORDINATED 

AUDIT 

AND 

APPRAISAL 

PROCESS 


The  activities  that  organization  need  to  perform  to  reduce  the  risk  inherent  in  process 
improvement  and  operations  in  multimodel  improvement  environments  are  outlined 
over  the  course  of  this  series  of  white  papers.  Given  that  an  organization  has  aligned 
the  improvement  effort  with  the  mission  and  business  objectives  of  the  organization, 
has  performed  a  systematic  approach  to  model  selection  and  composition,  and  has 
successfully  deployed  the  resulting  organizational  processes  with  aid  of  a  robust 
process  architecture,  there  are  still  barriers  to  reaping  the  full  potential  from  all  this 
effort.  One  of  the  most  significant  barriers  manifests  itself  out  of  the  cost  to,  and 
disruption  of,  operational  activities  resulting  from  the  requirement  to  demonstrate 
compliance  to  the  multitude  of  source  technologies,  now  embedded  in  the 
organizations  own  implemented  process.  Many  technologies  come  with  an  obligation 
to  appraise  or  audit  against  that  specific  technology  standard  (to  satisfy  legal 
requirements,  maintain  certification  or  to  identify  improvement  opportunities). 
Consequently,  the  number  of  different  types  of  audits  and  appraisals  organizations 
need  to  perform  are  growing.  Organizations  therefore  need  to  look  for  approaches  to 
performing  the  needed  audits  and  appraisals  more  efficiently  and  to  look  for  ways  to 
combine  audits  and  appraisals  (or  re-use  results  across  technologies)  in  order  to 
achieve  significant  cost  reductions.  Some  organizations  have  made  significant 
progress  in  performing  appraisals  in  a  more  efficient  manner,  indeed  using  one 
improvement  technology  to  improve  the  appraisal  methodology  of  another  [Hefner 
2001]. 

Let  us,  for  a  moment,  consider  the  circumstances  prevalent  in  the  automobile 
industry  in  respect  to  audits  and  appraisals.  This  industry  has  evolved  a  structure 
where  a  relatively  small  number  of  car  manufacturers  draw  the  components  and 
expertise  for  the  products  they  integrate  on  the  assembly  line  from  a  somewhat  larger 
group  of  direct  suppliers.  These  direct  suppliers  rely  in  turn  on  components  and 
expertise  from  second-  and  third-tier  suppliers.  This  structure  results  from  the  desire 
on  the  car  manufacturer’s  part  to  reduce  significantly  the  number  of  suppliers  they 
interact  with  directly.  This  has  the  consequence  that  the  car  manufacturers,  and 
indeed  the  direct  suppliers,  have  a  greater  reliance  on  their  suppliers  for  essential 
components.  As  image  is  so  essential  to  the  appeal  of  a  car  manufacturer  in  its 
market,  the  impact  poor  quality  in  the  supply  chain  can  have  on  the  image  of  a  car 
manufacturer  is  critical.  In  response  to  this  risk,  car  manufacturer,  and  in  turn,  direct 
suppliers,  have  turned  to  a  series  of  technologies  including  CMMI,  SPICE,  Auto- 
SPICE,  Lean,  Six  Sigma  and  others  to  provide  reassurance  that  the  organizations 
from  which  they  source  critical  components  have  a  demonstrable  level  of 
competence  and  capability  in  the  multitude  of  expertise  areas  they  require.  Thus, 
both  car  manufacturers  and  the  direct  suppliers  have  imposed  requirements  for 
process  capability,  based  on  a  series  of  different  improvement  technologies  on  their 
suppliers. 

In  order  to  verify  that  the  suppliers  indeed  have  the  levels  of  process  excellence 
required,  both  the  imposers  of  the  improvement  technologies  and  the  users  engage  in 
activities  to  demonstrate  compliance  to  the  required  standards.  These  activities  may 
take  the  form  of  SCAMPI  A,  B,  and  C  appraisals,  SPICE  assessments,  Auto-SPICE 
assessments,  ISO  audits  and  any  number  of  car  manufacturer  specific  audits,  to 
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mention  only  the  most  prevalent  types.  With  the  exception  of  the  final  category,  all 
these  activities  are  also  performed  because  of  internal  or  external  requests  for 
compliance.  The  results  of  all  these  very  commendable  activities  are  that  projects  and 
organizational  units  are  becoming  overwhelmed  with  appraisals,  audits  and 
assessments  of  internal  and  external  origin.  This  barrage  of  activities  has  a  serious 
impact  on  the  operations  of  these  companies  and  is  particularly  acute  where  key 
projects  are  working  on  critical  components.  The  result  is  that  these  projects  are 
overwhelmed  by  internal  and  external  appraisal  type  activities,  further  placing  these 
high  risk  projects  under  increased  schedule  pressure. 

There  are  several  potential  approaches  to  solving  the  problems  described  above  that 
also  offer  guidance  to  organizations  in  multimodel  environments  in  general.  They 
involve  addressing  the  innate  efficiency  of  any  individual  audit  event  as  well  as 
addressing  the  usage  of  single  audit  events  to  serve  multiple  models’  audit/ appraisal 
requirements.  The  approaches  address  different  aspects  of  improving  efficiency  and 
can  be  applied  internally  as  part  of  an  organization’s  multimodel  process 
improvement  effort.  To  effect  change  in  the  actual  appraisal  and  audit  methods, 
thereby  enabling  attainment  of  formal  audit  results  (not  just  “internal  results  and  gap 
analysis”)  in  a  more  efficient  manner,  organizations  and  interest  groups  can  work 
with  and  influence  the  model  and  standard  bodies  to  make  their  appraisal  and  audit 
approaches  more  interoperable  and  integrable. 

Efficiency  in  appraisals  and  audits,  and  related  factors  should  all  be  considered  when 
an  organization  using  multiple  process  improvement  models  makes  the  initial 
decisions  about  improvement  technology  selection.  Indeed  the  processes  used  to 
perform  mission  translation,  model  selection  and  model  composition  should  include 
criteria  that  help  select  model  combinations  that  are  more  compatible  when  process 
compliance  and  assurance  activities  need  to  be  performed  later.  Where  such  selection 
considerations  (i.e.,  audit  compatibility)  are  overridden  required  by  regulatory  or 
business  considerations,  the  improvement  organization  needs  to  give  early  and 
careful  consideration  to  how  it  can  achieve  internal  operational  efficiency  in  its 
execution  of  appraisals  and  audits  and  still  meet  improvement,  reporting  and 
compliance  objectives. 

As  stated  earlier,  there  are  costs  savings  to  be  realized  in  both  the  efficiency  of 
individual  audit  events  or  the  co-execution  of  appraisals  or  audits  internally. 
Efficiencies  in  single  events  may  be  attained  through  the  applications  of  such 
methods  as  Lean  to  the  appraisal  preparation  and  execution  [Hefner  2003],  [Hefner 
2004],  As  an  example  of  co-execution,  conducting  CMMI  and  ISO  9001:2000 
related  internal  appraisals/audits  by  integrated  or  cross-trained  teams  of  CMMI  and 
ISO  9001:2000  experts  is  a  feasible  and  practical  combination.  The  availability  of 
agreed  mappings  of  the  improvement  technologies  in  use  in  an  organization  to  the 
organizational  processes,  as  discussed  in  the  3ld  white  paper,  is  a  key  success  factor 
in  this  approach  and  facilitates  the  development  of  common  questionnaires  for 
independent  examination  of  compliance  in  projects  and  organizational  units.  Experts 
from  other  improvement  technologies  can  then  analyze  the  data  collected  by  the 
cross-trained  experts  thus  reducing  the  need  for  large  teams  of  auditors  with 
representatives  from  every  improvement  technology. 
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MEASUREMENT 


Just  as  many  of  the  pitfalls  of  process  improvement  in  multimodel  environments  may 
be  alleviated  through  appropriate  training  across  the  different  improvement 
technologies,  staff  involved  in  appraisals  and  audits  also  benefit  from  integrated  or 
cross  training.  This  involves  training  at  an  appropriate  level  in  other  technologies,  in 
the  difference  in  granularity  between  technologies  and  in  relevant  mapping  between 
the  technologies.  These  competencies  are  a  key  pre-requisite  for  the  co-execution  of 
appraisals  and  audits  and  to  a  mutual  acceptance  of  results  across  technologies.  Such 
cross  training  also  enables  innovations  that  may  be  required,  and  increases  credibility 
of  those  who  innovate.  For  example,  while  some  innovation  is  required,  it  is  possible 
to  combine  results  from  SCAMPI  appraisals,  ISO  audits  and  SPICE  appraisals  for 
example,  at  least  for  internal  evaluation,  strength  and  weakness  identification  and 
assessment  of  compliance. 

It  is  more  difficult  to  improve  efficiency  in  externally  required  appraisals  and  audits 
than  in  internally  driven  ones.  For  appraisals  and  audits  required  by  regulatory  or 
compliance  bodies  or  by  customers,  organizations  do  not  have  much  say  about  the 
type  and  frequency  of  evaluations  they  are  subject  to.  As  a  result,  organizations  that 
are  suffering  from  the  frequency  of  such  evaluations  turn  to  model  mappings  to  help 
them  demonstrate  compliance  from  evaluations  against  one  technology  against 
another.  While  this  is  a  help  internally  in  understanding  the  relationship  between  the 
technologies  they  are  using,  it  is  of  little  or  no  utility  if  the  evaluating  organization 
does  not  recognize  the  validity  of  the  mappings,  the  appraisal  methods  and  the 
results.  There  are  initiatives  stalling  to  form,  especially  in  the  automotive  industry,  to 
get  standards  bodies  to  look  at  possible  combined  appraisals,  mutual  or  unilateral 
recognition  of  other  appraisal  methods  and  indeed  the  sharing  of  appraisal  and  audit 
results  within  individual  industries,  where  the  industry  structure  encourages  such 
sharing.  Success  with  such  initiatives  will  however  depend  on  the  economic 
necessity  within  each  industry.  Where  the  economic  drivers  for  integration  and 
mutual  recognition  of  appraisal  approaches  is  great  enough,  industry  lobbyists  may 
be  able  to  exert  enough  pressure  on  the  standard  bodies  to  force  the  needed  change. 
Thus  the  solution  across  broader  industry  may  ultimately  rely  more  on  economic  and 
political  power  than  technical  knowledge. 


Many  improvement  technologies  explicitly  address  or  contain  measurement  within 
their  scope.  And,  while  harmonization  involve  reconciling  these  features  in  the 
composition  and  implementation,  measurement  infrastructure  in  fact  plays  a  much 
more  significant  role.  Measurement  activities  are  not  performed  to  satisfy 
improvement  technologies  requirements,  but  are  serve  as  a  governing  factor  that  both 
guides  and  motivates  improvement  in  the  organization.  In  fact,  measurement, 
working  closely  with  mission  translation  serves  as  an  integrating  and  harmonizing 
factor  across  improvement  technologies.  By  knowing  what  we  care  about  in  terms  of 
success  measures,  we  can  more  effectively  decompose  our  objectives  and  plans,  and 
align  our  improvement  projects.  In  practice,  we  can  better  prioritize  through 
hypothesizing  (or  we  have  the  data,  quantifying)  the  contribution  of  each 
improvement  to  the  overall  success  indicators.  This  helps  us  make  better  decisions 
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about  resource  allocation,  which  projects  to  do  and  perhaps  just  as  important,  which 
not  to  do.  The  measurement  function  in  a  harmonized  environment  should  be 
agnostic  about  the  technologies  contributing  any  particular  metric — the  focus  is  on 
the  results  achieved. 

The  measurement  activities  in  a  harmonized  improvement  environment  also  need  to 
measure  the  effectiveness  of  the  harmonization  activities.  We  need  to  be  able  to 
quantify  whether  the  expected  benefits  of  harmonization  are  being  realized.  The 
benefits  we  need  to  measure  are  often  complex  may  include: 

•  Cost  reduction  through  economies  of  scale  for  all  aspects  of  model 
implementation 

•  Cycle -time  reduction  for  improvement  efforts  and  the  realization  of  performance 
objectives 

•  Culture  change  related  to  establishment  of  enterprise  processes,  measurement 
systems,  and  more 

•  Process  robustness  to  an  ever-evolving  and  dynamic  world  of  models  and 
regulations 

•  Long-term,  robust,  and  effective  organizational  approach  to  technology  and 
model  selection 

•  Ability  to  deal  effectively  with  different  structures  and  terminology  of 
implemented  models 

•  Cost  reduction  in  relation  to  audits  and  assessments  for  operational  units  and 
projects 

Economies  of  scale  have  also  to  be  achieved  for  the  measurement  infrastructure  in 
the  harmonized  organization.  As  indicated  in  the  first  paragraph  on  measurement, 
almost  every  improvement  technology  contains  some  form  of  measurement 
component.  We  absolutely  need  to  avoid  establishing  as  many  measurement 
initiatives  and  approaches  as  we  have  improvement  technologies.  In  a  harmonized 
approach  the  creation  of  a  integrated  measurement  infrastructure  that  supports  and 
enables  effective  measurement  across  multiple  improvement  is  a  critical  and  cost 
effective  task.  The  cultural  benefits  an  organization  derives  from  a  harmonized 
measurement  infrastructure  should  also  not  be  underestimated.  Having  a 
measurements  focus  as  an  integral  part  of  your  product  and  process  development 
activities  fosters  the  quantitative  approach  to  reasoning  in  regard  to  problem  solving 
in  product  and  process  design  that  is  a  key  element  of  Six  Sigma  and  the  high 
maturity  practices  of  the  Software  Engineering  Institutes  CMMI. 

The  actual  measurement  infrastructure  installed  in  an  organization  will,  of  necessity, 
be  influenced  by  the  improvement  technologies  selected  for  implementation  in  the 
organization.  The  implementation  of  CMMI  in  an  organization,  for  example,  will  be 
strongly  influenced  by  a  concurrent  implementation  of  Six  Sigma,  if  a  harmonized 
approach  is  followed.  Six  Sigma  provides  outstanding  analysis  methods  (it  presumes 
there  is  already  measurement),  and  in  non-manufacturing  settings,  establishing  a 
measurement  infrastructure  that  often  must  be  an  implemented  as  an  “enabling 
project”,  providing  capabilities  to  other  improvement  project  later.  Six  Sigma 
measurements  capability  can  thus  be  “baked  into”  the  CMMI  practices  and  may  even 
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be  used  to  improve  the  improvement  effort  overall  (see  Siviy  2007  for  a  lot  of  detail 
on  this  particular  combination).  In  the  software  engineering  world,  we  also  have 
Goal-Question-Indicator-Metric  (GQIM)  and  Practical  Software  and  Systems 
Measurement  (PSM).  Both  are  effective  for  establishing  measurement  infrastructure. 
GQIM  in  particular  has  a  front-end  that  involves  goal  decomposition — making  it  a 
natural  partner  of  the  whole  mission  translation  activities.  Also,  GQIM  has  the 
different  indicator  types,  making  it  easy  to  see  how  to  map  measures  to  goals, 
strategies  and  tactical  plans.  As  such,  it  transcends  software  engineering. 

As  stated  in  the  white  paper,  Maximizing  your  Process  Improvement  ROI  through 
Harmonization,  from  our  research  observations,  the  most  successful  organizations 
using  multiple  improvement  technologies  create  a  process  architecture  and 
accompanying  process  descriptions — their  “corporate  way” — and  then  map  the 
technologies  of  interest  to  it.  This  implemented  process  is  the  primary  focus  of  the 
measurement  infrastructure  in  a  harmonized  approach,  as  this  process  is  the  one  used 
by  the  organization  to  implement  the  activities,  which  deliver  on  the  mission,  and 
goals  established  at  strategic  levels.  The  measurement  system  adjustments  needed  for 
a  harmonized  improvement  approach  should  be  minor.  If  an  organization  has 
harmonized  their  approach  to  improvement,  the  measurement  system  should  now 
also  support  the  organization  in  determining  whether  the  improvements  implemented 
in  the  organization  are  leading  to  mission  fulfillment  and  goal  achievement  at  the 
strategic  level  and  deliver  the  operative  units  and  projects  with  the  information  to 
support  the  day-to-day  running  of  the  operation. 


FUTURE 

RESEARCH 


Process  improvement  groups  in  the  context  of  multimodel  improvement  face  many 
implementation  challenges  separate  and  distinct  from  single  model  improvement. 
While  this  white  paper  identifies  an  approach  to  working  through  these  challenges, 
we  realize  more  research  is  needed  in  this  area  to  be  able  to  confidently  address  the 
following  questions: 


•  What  is  our  mission?  What  are  our  goals?  Are  we  achieving  our  goals?  What  stands  in  our  way? 

•  Which  organizational  structures  best  support  harmonized  process  improvement  in  an  organization? 

•  Are  there  organizational  structures  that  better  suit  specific  improvement  technology  combinations? 

•  What  are  the  advantages/disadvantages  of  different  organizational  structures  concerning  the  support  they  offer  to  a 
harmonized  approach? 

•  How  do  we  support  /enable  effective  cooperation  and  coordination  across  different  organizational  improvement  technologies? 

•  What  are  effective  approaches  to  integrated  training  and  cross-training? 

•  What  are  the  characteristics  and  best  practices  associated  with  an  effective  improvement  project  portfolio  management? 

•  How  do  we  achieve  and  maintain  cost-effectiveness  and  cost-reduction  in  regard  to  the  multiple  audits  and  appraisals 
organization  have  to  conduct? 

•  How  can  mutual  acceptance  of  audit  and  appraisal  result  be  achieved  in  industries  where  it  has  become  an  acute  cost  and 
productivity  factor?  (How  can  the  standards  bodies  be  influenced  to  support  cross  standard  efforts?) 

•  What  combination  of  technologies  enable  synergy  between  the  measurement  infrastructures  of  the  individual  technologies  ? 

•  What  strategies  and  approaches  offer  possible  best  practice  for  harmonizing  measurement  infrastructures? 
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